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DETAILED ACTION 

This office action is in response to the Appeal Brief filed 8/21/2009, in which 
claims 1-21 were presented for appeal. 

The finality of the rejection of the last Office action is withdrawn. 

Response to Arguments 

Applicant's arguments with respect to claims 1 and 13 have been considered but 
are moot in view of the new ground(s) of rejection. The newly applied reference 
Rosenberg more fully explains the known application of overlaying tactile sensations by 
an operator's computer from stored files. Rosenberg explains this feature does not 
teach away from the applications of allowing the user to feel the remote location 
accurately, such as in remote surgery, because it can reduce the mental tasks of the 
user. 

Regarding, Applicant's arguments that the applied references do not teach 
calibrating actuators in connection with variable sensitivities associated with different 
regions of the user, have been fully considered but they are not persuasive. Simmons in 
view of Mcintosh describes calibrating the actuators to various sensitivities associated 
with the human body would allow the user to determine how much or little of the remote 
environment they wish to feel. 

Regarding Applicant's arguments that the applied references do not teach 
actuators on the robot corresponding to motion sensors on the user's body suit, the 
Examiner disagrees and refers Applicant to figure 2 and the corresponding description. 
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Regarding claims 6 and 17, Applicant arguments against the references 
individually; merely that Abbasi does teach each and every limitation. However, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). It is the combination of all references that teach and make obvious the claimed 
invention, not Abbasi alone. 

Claim Rejections - 35 USC § 103 

The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Claims 1-4, and 13-15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons, US 6,741 ,91 1 , in view of Mcintosh, US 5,1 03,404, in 
further view of Rosenberg, "The Use of Virtual Fixtures as Perceptual Overlays to 
Enhance Operator Performance in Remote Environments" AL/CF-TR-1 994-0089, 
published in DTIC 1995. 

(Claims 1 and 13) Simmons describes a virtual reality encounter system and 
method comprising: A humanoid robot having tactile sensors positioned along the 
exterior of the robot (column 8, lines 39-50), the sensors sending tactile signals to a 
communications network (column 7, lines 29-32); and a body suit having tactile 
actuators (column 6, lines 33-51, column 8, lines 39-50), the actuators receiving the 
tactile signals from the corresponding tactile sensors on the robot from the 
communication network (column 7, lines 29-32), wherein the tactile sensors and the 
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corresponding tactile actuators are calibrated in connection with variable sensitivities 
associated with different regions of the human (column 8, line 62- column 9, line 4, 
column 13, lines 3-28). Further Mcintosh teaches that it was well known in the remote 
robotic control art to calibrate sensors to different levels of sensitivities to overcome the 
problem and allow, according to Mcintosh, individuals to vary the sensitivities of tactile 
feedback to optimize their own degree of sensitivity and control over the manipulator 
(column 1, lines 41-50, column 9, lines 12-52). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to calibrate the tactile actuators of the 
body suit, because as Mcintosh suggests this would allow the user to preset and vary 
the sensitivities of the suit actuators in individual body locations to the user's preferred 
degree of sensitivity of how to feel the remote environment. This would as Simmons 
suggests, allow the user to feel pressure or force that is proportional to the robot at 
remote locations (column 40, lines 39 -45), allowing the user's hands to feel more 
delicate level of sensation or allowing the user to feel superhuman, by feeling only a 
little of a large force (column 40, lines 19-64). 

Simmons further describes the user apparatus overlays supplemental tactile 
sensations from stored virtual tactile sensations that are sent to the body suit (column 8, 
line 62-column 9, line 4, and column 12, lines 56-65) in the form "[user] feels the rocks 
holding him up and the wind shaking his body" in a virtual world that does not exist. 
Further, Rosenberg teaches the overlaying of virtual tactile sensations by an operator's 
or host computer was well known in the remote robotic art. Rosenberg teaches an 
operator's computer overlaying virtual fixtures that provide "abstract sensory information 
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overlaid on top of reflected sensory feedback from a remote environment" (page 3, lines 
12-17), wherein the "[a]bstract fixtures could be composed of visual, auditory, even 
tactile sensations used alone or in cross-modal combinations" (page 4, lines 13-14). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to 
combine the invention of Simmons in view of Mcintosh with the teachings of Rosenberg 
because, as Rosenberg suggests the additional virtual tactile sensations "act to reduce 
mental processing required to perform the task, reduce the work load of certain sensory 
modalities, and most of all allow precision and performance to exceed natural human 
abilities." (Rosenberg: page 3, lines 18-21.) 

Rosenberg clearly states the overlays are applied at the operator's computer 
station: "virtual fixtures are completely independent of all information from the remote 
site and are thus immune from communication delays and bandwidth limitations" (page 
3, lines 15-17). Rosenberg does not describe the operator's computer that overlays the 
virtual fixtures as a gateway device. However, it was well known in the computer arts 
that a gateway can be embodied in a general computer to perform the protocol 
conversion or the functions that a separate gateway device (such as a router or modem) 
performs. It would have been very obvious to one of ordinary skill in the communication 
arts to utilize a general computer with an internal gateway device (internal router or 
modem) as the operator's computer which overlays the virtual fixtures, as this would 
have been merely a design choice of the network system, comprised of well known 
network components. 
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(Claims 2 and 14) Simmons further describes motion sensors positioned 
throughout the body suit (column 13, lines 50-67), the motion sensors sending motion 
signals corresponding to movements of each sensor relative to a reference point 
(column 14, lines 30-41), the motion signals transmitted to the communications network 
(column 7, lines 29-32); and the humanoid robot, receiving, from the communications 
network the signals from the motion sensors (column 11, lines 15-60), the signals from 
the motion sensors causing a movement of the robot that is correlated to a movement of 
the body suit (column 8, lines 23-30). 

(Claims 3 and 15) Simmons further describes wherein the robot includes 
actuators corresponding to the motion sensors, the actuators causing the robot to move 
(figure 2). 

(Claim 4) Simmons describes the robot comprising a body (column 5, 40-58); a 
camera coupled to the body, the camera for sending video signals to the 
communications network (column 7, lines 2-8, column 11, lines 51-52); and suggests 
sending audio information to the local site over the communications network (column 
1 1 , lines 51-52) further describing that a sound sensor means is coupled to second 
body (robot) to capture sound for sending audio signals to the communications network 
(column 4, lines 20-28 and claim 49). 

Claim Rejections - 35 USC § 103 

Claims 5, 8 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, and Rosenberg as applied to claims 4 and 14 
above, and further in view of Simmons US 20030030397 (Simmons '397). 
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(Claims 5, 8, and 16) Simmons describes wherein the user wears a "wrap 
around video display or a holographic display over his eyes" to render the video signals 
received from the camera (column 9, lines 5-10) and a transducer to transduce the 
audio signals received from the sound sensor coupled to the robot (claim 49, column 4, 
lines 20-28). Simmons '397 teaches that the robot would comprise microphones in the 
appropriate ear location to be relative to the ear position of the user (claim 19) and that 
the head display would comprise of goggles or glasses (claim 1 2). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Simmons '397 with the invention of Simmons, in view of Mcintosh and 
Rosenberg, because as Simmons '397 suggests, the microphones and the goggles are 
one well known means of creating the 360-degree, stereoscopic, realistic feedback of 
the remote location ([0016]). Simmons further describes wherein the communications 
network comprises an interface having one or more channels for receiving the audio 
signals form the sound sensors (column 7, lines 29-32); receiving the video signals form 
the camera (column 7, lines 2-8); sending the video signals to the head display (column 
9, lines 5-10); and sending the audio signals to the sound producing means (column 4, 
lines 20-28). 

(Claims 9 and 18) Simmons further describes wherein the body includes an eye 
socket and the camera is positioned in the eye socket (column 7, lines 2-8). 

Claim Rejections - 35 USC § 103 

Claims 11, 12, 20 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons in view of Mcintosh, and Rosenberg as applied to claims 5, 
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Simmons in view of Mcintosh, Rosenberg, and Algazi as applied to claim 10, and 
Simmons in view of Mcintosh, Rosenberg, and Simmons '397 as applied to claim 16, 
further in view of Yee, US 6,016,385. 

Simmons in view of Simmons '397 describes transmitting video signals to a set of 
goggles but does not specify a receiver or that the data is transmitted wirelessly. 
However, Yee teaches a headset of a user comprises a receiver to receive video 
signals (column 5, lines 11-37) and wherein the robot comprises a transmitter to 
wirelessly send the audio, tactile, motion and video signals to the communications 
network (communications antenna 30). It would have been obvious to one of ordinary 
skill in the art at the time of the invention to combine the invention of Simmons in view of 
Mcintosh, Rosenberg and Simmons '397 with the teachings of Yee because it was well 
known that wireless means would offer the predictable result of more accessible travel 
of the robot and a wider range of motion. 

Claim Rejections - 35 USC § 103 

Claims 10 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg and Simmons '397 as applied to claims 
5 and 16 above, and further in view of Algazi, US 7,333,622. 

Simmons and Simmons '397 describe wherein the sound sensor is positioned on 
the robot relative to the position of the sound receiver on the person and can be refined 
by the shape of the outer ear (column 4, lines 20-28). Algazi teaches it was well known 
in the art to place listening devices in a mannequin having the exact size, shape, and 
acoustic properties of the listener located in the ear canals to replicate the sound 
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signals accurately (column 3, lines 22-52). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to combine the teachings of Algazi 
with the invention of Simmons in view of Mcintosh, Rosenberg and Simmons '397 
because Algazi describes merely one means of achieving the result described by 
Simmons of replicating the 360 degree, precise audio feedback to the remote user. 
Claim Rejections - 35 USC § 103 

Claims 6, 7 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg, Simmons '397 and Yee as applied to 
claims 5 and 16 above, and further in view of Abbasi, US 6,786,863. 

Simmons describes a remote robot operating system and method wherein one 
user controls a robot that duplicates the actions of the operator and transmits the 
sensed condition of the robot environment to the operator, wherein the sensed 
environment is overlaid visually and reproduced by actuators and sensors on the 
operator (column 1 , line 36-column 2, line 12). Simmons does not describe wherein at 
the location of the operator, a second humanoid robot transmits data to a first location. 
However, Abbasi teaches this duplication of the same system to create an interaction 
between remote users is known. Abbasi teaches a remote physical contact system and 
method wherein a first surrogate (robot) is at a first location, a second surrogate (robot) 
is at a second location, the second surrogate having the same components, actuators, 
and sensors, i.e. a second microphone and second camera (figure 1, elements 35B, 
40B, and 45B); a second display to receive the video signals from a first camera, a 
second earphone to receive audio signals from a first microphone (figure 1 , elements 25 
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and figure 6), And a first communication gateway in the first location and a second 
communication gateway in the second location to create the remote interaction via a 
network (computer network 30 between computers 15 and 25). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Abbasi with the invention of Simmons, in view of Mcintosh, Rosenberg and 
Simmons '397, because as Abbasi teaches the use of remote surrogates and expands 
the notion by using dual surrogates for teleconferencing or computer communications, 
adding a capability to engage in all types of physical contact to "provide for the tactile 
sensation so inherent in many forms of human contact." (Column 1, lines 44-64.) 
Further the combination of the prior arts would produce a predictable result by merely 
duplicating the known systems and interchanging the physical locations, as clearly 
suggested by Abbasi. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Burdea et al., US 5,354,162, teaches an actuator system to provide force 
feedback to a glove worn by a user, the glove attached to a host computer, wherein the 
host computer "acts as a gateway to transmit instructions to a robot hand 18 in an 
environment" (column 6, lines 59-63). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTINE M. BEHNCKE whose telephone number is 
(571)272-8103. The examiner can normally be reached on 8:30 am- 5pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Thomas G. Black can be reached on (571) 272-6956. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

CMB 

/Thomas G. Black/ 

Supervisory Patent Examiner, Art Unit 3661 



